Automated languages phylogeny from Levenshtein distance

Languages evolve in time according to a process in which reproduction, mutation and extinction are all possible. This is very similar to haploid evolution for asexual organisms or for mtDNA of complex ones. Exploiting this similarity it is possible, in principle, to verify hypotheses concerning their relationship. The key point is the definition of the distance among pairs of languages in analogy with the genetic distance among pairs of organisms. Assuming that vocabulary is the analogue of DNA, distances can be evaluated from lexical differences. This concept seems to have its roots in the work of the French explorer Dumont D’Urville. He collected comparative words lists of various languages during his voyages aboard the Astrolabe from 1826 to 1829 and, in his work about the geographical division of the Pacific, he proposed a method to measure the degree of relation among languages. The method used by modern glottochronology, developed by Morris Swadesh in the 1950s [20], measures distances from the percentage of shared cognates, which are words with a common historical origin. The weak point of this method is that subjective judgment plays a relevant role. In fact, even if cognacy decisions are made by trained and experienced linguists, the task of counting the number of cognate words in a list is far from being trivial and results may vary for different studies. Furthermore, these decisions may imply an enormous working time. Recently, we proposed a new automated method which has some advantages: the first is that it avoids subjectivity, the second is that results can be replicated by other scholars assuming that the database is the same, the third is that no specific linguistic knowledge is requested, and the last, but surely not the least, is that it allows for rapid comparison of a very large number of languages. The distance between two languages is defined by considering a renormalized Levenshtein (or edit) distance among words with the same meaning and averaging on the words contained in a list of 200 features. The renormalization, which takes into account the words length, plays a crucial role, and no sensible results can be found without it. Assuming a constant rate of mutation, these lexical distances are logarithmically proportional, in average, to the genealogical ones. Then, phylogenetic trees can be reconstructed from the matrix which contains the genealogical distances between all pairs of languages in a family. We applied our method to the Indo-European and to the Austronesian groups considering, in both cases, fifty different languages. From the two matrices, we obtained two genealogical trees using the Unweighted Pair Group Method Average (UPGMA) [19]. The trees are similar to those found by previous research [6, 7] with some important differences concerning the position of few languages and subgroups. Indeed, we think that these differences carry some fresh information about the structure of the tree and about the phylogenetic relations inside the families. Automated languages phylogeny from Levenshtein distance 2